There are three main materials for drills, mills, and other cutting tools, listed from softest to hardest, and not coincidentally from least to most expensive: High Speed Steel (HSS), Cobalt, & Carbide. I recommend that you have a small set of Cobalt drills and mills, filled out with (as) full (as you can afford) sets in HSS. For instance, I have a 1/32” to 1/2” Cobalt drill set plus a big box of less expensive HSS numbered, fractional, and lettered drill bits (also up to 1/2”). Once you dull or break the ones you use the most, say the tap and clearance drill sizes, you can replace them with better.
The same can be said for milling cutters, although an
extensive selection of diameters is probably not so important –
except for special cases like nice square countersink holes. There
are special 2-flute styles for Aluminum which are much less likely to
gum up, but otherwise 4-flute cutters are probably fine. I'd get a
couple sizes of “rouging” 4-flute cutters that will save you
breaking the regular ones. Also, ball end (rounded), angled
(for dovetail slots), and chamfered (for rounding edges) can be handy.
Then some different sizes and angles of countersinks, which can also be used to debur holes. The standard flat-head screw is 82º but having some 60º and 90º ones can be useful.
Also spotting and centering drills to get holes started accurately, I like these: https://www.mscdirect.com/product/details/71332241 & https://www.mscdirect.com/product/details/01035054 (there are some of this type with 82º countersinks which makes repeated hole drilling in metal sheet much quicker).
I have a largeish set of Silver and Demming drills (1/2”
reduced shaft) from 9/16” to 1-1/2” that come in handy sometimes,
but also, being only HSS, get dull easily as well, so YMMV.
I also have a bunch of reamers of all dimensions, fractional, over-under, metric, special like 27/32" that is the clearance for 1/2” (US) plumbing pipe, and others that are of no real use, like a set of adjustable hand reamers that I have used twice and broken once. If you are going to spend money on this stuff, get the standard metric and SAE sizes that should fit your motor shafts and bushings.
Instead, get the biggest set of SAE and Metric taps and dies that you can afford. You can replace the broken and dull ones with better quality as you go along...
In my cutter and collet drawer I have an offset boring head and bar set that is sometimes handy, and some fly cutters which I've hardly ever used. There is also a slitting saw arbor that can hold things like this: https://www.mscdirect.com/product/details/73322042, which is quite a bit more dangerous than it looks; and, an MT2 to R8 adapter in case you have some of those old-fashioned drills.
A set of R8 collets, by 1/8” if you can, to hold different sizes of mills and one or two R8 mounted drill chucks are required. I have 1/2” keyed and (a medium expensive) 3/8” keyless chucks. The keyless one is better quality, so centers better, and is invaluable when stepping sizes of holes or switching from drill to tap, over and over.
The whole set of T-nuts and 1/2” studs with step-block clamps like this: https://www.mscdirect.com/product/details/06944540, is absolutely necessary. You can fill it out with 1/2” washers and threaded rod for larger and smaller uses. There are other varieties of hold-downs and clamps that can be used to drain your funding accounts, so graze around in the catalogs. I've also drilled a couple heavy steel bars to I can use them as straps over and across awkward parts.
Other fixturing utilities are 1-2-3 blocks,
to keep things level and even, a 90º angle plate
to keep things square, and V-blocks
with their own clamps, to hold round stock solidly.
I have parallels, for leveling things
in a vise -- but tend to use blocks of soft wood or plywood
which seem to be parallel-enough(TM).
My go-to holder is a cheap 6” drill press vise
that I machined flat and square (that's actually a use for the fly
cutters, eh?). I squared one edge to the back jaw so I can quickly set
it up with a normal machinist square. A “real” machinist's vise
is quite a bit more expensive, but probably better in every
dimension.
I also have a gigantor 6” jaw swivel vise which
I got cheap at Harbor Freight because they didn't know what to do
with it. It adjusts angles horizontally and vertically and comes in
handy when I need a more solid clamping. I am just about at the end of my
time being able to lift the danged thing, but here's a smaller one at MSC:
https://www.mscdirect.com/product/details/70038153
The usual combination square (like this, but get a good set: https://www.harborfreight.com/12-in-combination-square-set-63688.html) with angle and centering heads gets used constantly. I use my g.Father's depth gauge (like: https://www.grizzly.com/products/Grizzly-Vernier-Depth-Gauge-6-x-1-128-/G9287) for just about everything, such as setting up offsets and lengths. And, of course, a good dial or digital caliper. Also a feeler gauge with lots of leaves is good for checking those tiny clearances.
These two, a center finder and an edge finder with both cone and cylinder ends, are really handy for setup – you can use the edge-cone with the rotary table drill arbor to get centered pretty good. This is the first page that came up when I looked for center finder: https://www.bearingtoolscentre.com/product/center-finder-assembly/ so don't take it as a recommendation.
For accurate setup these dial and test gauges are very useful,
and the bracket shown in the photo makes it easy to attach them to the
mill quill to check position and travel, or get the vise really square
to the table travel.
I point out my hand-calipers and dividers which are useful in
various pinches, old school but also not very expensive.
Then you can just go wild with other stuff, like micrometers, centering gauges, angle plates and sine bars, and Guadalupe-knows what else...
Since I live in a (material) desert I find it very useful to have a selection of hardware -- big boxes of machine screws, i.e., nuts and bolts, in many sizes. For instance, this: https://www.mscdirect.com/product/details/67758680, has been INVALUABLE in greasing the forward progress of many projects.
And a large selection of round stock. You can usually find
plates and angles, but round stuff of the right material is harder.
I get cold rolled and oil-hardening tool steel rods,
steel and brass tubes and pipes, some bits of aluminum rod, and lengths
of acrylic, delrin, and nylon plastic rods whenever I order other
materials, just to have them around when needed.